Visually Grounded Meaning Representations
نویسندگان
چکیده
منابع مشابه
Learning Visually Grounded Sentence Representations
We introduce a variety of models, trained on a supervised image captioning corpus to predict the image features for a given caption, to perform sentence representation grounding. We train a grounded sentence encoder that achieves good performance on COCO caption and image retrieval and subsequently show that this encoder can successfully be transferred to various NLP tasks, with improved perfor...
متن کاملLearning Grounded Meaning Representations with Autoencoders
In this paper we address the problem of grounding distributional representations of lexical meaning. We introduce a new model which uses stacked autoencoders to learn higher-level embeddings from textual and visual input. The two modalities are encoded as vectors of attributes and are obtained automatically from text and images, respectively. We evaluate our model on its ability to simulate sim...
متن کاملImproving Visually Grounded Sentence Representations with Self-Attention
Sentence representation models trained only on language could potentially suffer from the grounding problem. Recent work has shown promising results in improving the qualities of sentence representations by jointly training them with associated image features. However, the grounding capability is limited due to distant connection between input sentences and image features by the design of the a...
متن کاملDeriving continous grounded meaning representations from referentially structured multimodal contexts
Corpora of referring expressions paired with their visual referents are a good source for learning word meanings directly grounded in visual representations. Here, we explore additional ways of extracting from them word representations linked to multi-modal context: through expressions that refer to the same object, and through expressions that refer to different objects in the same scene. We s...
متن کاملPerceptually grounded meaning creation
The paper proposes a mechanism for the spontaneous formation of perceptually grounded meanings under the selectionist pressure of a di~rimination task. The mechanism is defined formally and the results of ,~me simulation experiments are reported.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Pattern Analysis and Machine Intelligence
سال: 2017
ISSN: 0162-8828,2160-9292
DOI: 10.1109/tpami.2016.2635138